Media adaptation determination for wireless terminals

ABSTRACT

A method (and corresponding equipment) by which a multimedia message is sent from a sending terminal ( 21 ) via a messaging server ( 22 )—such as a MMS Proxy-Relay in MMS or a SIP proxy server in SIP IM—to a receiving terminal ( 25 ) having limited multimedia capabilities, with the sending terminal ( 21 ) adapted to include a user agent ( 21   a ) for inserting, into the message, media characteristics of the message sufficient in detail to enable the messaging server ( 22 ) to determine whether the message should be transcoded based on actual or assumed multimedia capabilities of the receiving terminal ( 25 ), and with the messaging server ( 22 ) configured to read the media characteristics and decide whether the message should be transcoded based only on the inserted media characteristics and on actual or assumed multimedia capabilities of the receiving terminal ( 25 ). The media characteristics are advantageously inserted into the header of the message.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application is a continuation application of U.S.Publication No. 2005/0165913 published on Jul. 28, 2005 (U.S. patentapplication Ser. No. 10/765,576 filed on Jan. 26, 2004). The subjectmatter of the previously filed application is hereby incorporated byreference.

BACKGROUND

The invention relates to the field of media adaptation in MultimediaMessaging Service (MMS) and SIP (Session Initiation Protocol) InstantMessaging (IM) to enable interoperability. More particularly, thepresent invention pertains to using signaling to determine whether mediaadaptation (transcoding) for a telecommunications terminal must beperformed.

Terminals available for multimedia messaging and so-called instantmessaging have different media and network capabilities (e.g. differentmedia formats supported, different limitation in message size and imageresolution, etc.). A messaging server, such as an MMS Proxy-Relaylogical entity as defined in the Open Mobile Alliance (OMA)specifications of the Multimedia Messaging Service (MMS) or a SIP proxyin SIP IM, has no standard way of knowing in advance whether mediaadaptation—called here transcoding (to accommodate limited multimediacapabilities of the receiving terminal)—is needed or not when providinga multimedia message to a terminal. Today a server must analyze allcomponents of a message in view of the capabilities of the terminal towhich the message is being sent, which means that a message must beopened to determine its relevant media characteristics.

The relevant characteristics of a message include for example: imageresolution, whether a JPEG (Joint Photographic Experts Group) isbaseline or progressive, and the number of frames of a GIF (graphicsinterchange format) image. If the message has multiple components (forexample multiple JPEG images or multiple images of different formats)then the server has to analyze all the components of the message. Forsome server implementations, such an analysis requires that for eachmedia component, a processing component (e.g. a plugin) supporting themedia type of the component be identified, and that the media componentthen be sent to the identified media processing component for analysis.In the simplest cases, the analysis can be performed by decoding onlythe beginning (e.g. header part) of the media component. Nevertheless,the analysis requires processing in respect to each media component andrequires the presence of a processing component capable of the analysis.The performance penalty can be very high if the transcoding engine islocated on a separate server (e.g. in case of one vendor providing themessaging server but opting to use a transcoding server provided byanother vendor, as illustrated in FIG. 1).

Thus, it would be advantageous to have a mechanism by which a messagingserver can determine whether transcoding/media adaptation is needed fora message intended for a terminal without having to (open and) examineeach media component of the message.

SUMMARY

Accordingly, in a first aspect of the invention, a method is provided bywhich a multimedia message is transcoded en route from a sendingterminal via a messaging server to a receiving terminal having limitedmultimedia capabilities, so as to be suitable for reception andpresentation by the receiving terminal, the method characterized by: astep in which a user agent of the sending terminal inserts, into themessage, media characteristics of the message sufficient in detail toenable determining whether the message should be transcoded toaccommodate multimedia capabilities of the receiving terminal; and astep in which the messaging server reads the media characteristics anddecides whether the message should be transcoded based only on theinserted media characteristics and on actual or assumed multimediacapabilities of the receiving terminal.

In accord with the first aspect of the invention, the messaging servermay send the message to a transcoding server if transcoding is needed,and the transcoding server may use the inserted media characteristics toitself decide if transcoding is needed.

Also in accord with the first aspect of the invention, the messagingserver may send the message to a transcoding server if transcoding isneeded, and the transcoding server may use the inserted mediacharacteristics to itself decide which parts of the message needtranscoding.

Also in accord with the first aspect of the invention, the messagingserver may determine, from the inserted media characteristics, whichparts of the message need transcoding and may send the message to atranscoding server if transcoding is needed for any message part, andmay include in the message an indication of which parts of the messageneed transcoding.

Also in accord with the first aspect of the invention, the messagingserver may determine, from the inserted media characteristics, whichparts of the message need transcoding and may send only those messageparts requiring transcoding to a transcoding server.

Also in accord with the first aspect of the invention, the method may befurther characterized by: a step in which transcoding is performed basedon the inserted media characteristics and the actual or assumedmultimedia capabilities of the receiving terminal, without performing ananalysis of the message to determine whether transcoding is needed.Further, in the step in which transcoding is performed, the transcodingmay be performed without also performing even an analysis to determinewhich parts of the message need to be transcoded.

Also in accord with the first aspect of the invention, the user agentmay insert the media characteristics into a field in the header of themessage.

Also in accord with the first aspect of the invention, the user agentmay insert the media characteristics into a header field in the body ofthe message.

Also in accord with the first aspect of the invention, the mediacharacteristics may include image and video resolution, or number offrames and frame rate of visual content, or sampling rate of audiocontent.

A second aspect of the invention provides a sending terminal, adaptedfor sending a multimedia message via a messaging server to a receivingterminal having limited multimedia capabilities, the sending terminalcharacterized by: a user agent for inserting, into the message, mediacharacteristics of the message sufficient in detail to enable themessaging terminal to determine whether the message should be transcodedbased only on actual or assumed multimedia capabilities of the receivingterminal and the inserted media characteristics.

A third aspect of the invention provides a messaging server, enhancedfor determining whether to transcode a multimedia message sent from asending terminal to a receiving terminal having limited multimediacapabilities, the messaging server characterized by: a characteristicsreader and analyzer, responsive to the message, for deciding whether themessage should be transcoded based only on comparing mediacharacteristics inserted into the message with actual or assumedmultimedia capabilities of the receiving terminal.

A fourth aspect of the invention provides a system, comprising a sendingterminal and a messaging server, both adapted to perform according to amethod by which a multimedia message is transcoded en route from thesending terminal via the messaging server to a receiving terminal havinglimited multimedia capabilities, so as to be suitable for reception orpresentation by the receiving terminal, the system characterized inthat: the sending terminal includes a user agent for performing a stepof inserting, into the message, media characteristics of the messagesufficient in detail to enable determining whether the message should betranscoded to accommodate multimedia capabilities of the receivingterminal; and the messaging server includes means for performing a stepof reading the media characteristics and deciding whether the messageshould be transcoded based only on the media characteristics and onactual or assumed multimedia capabilities of the receiving terminal.

In accord with the fourth aspect of the invention, the messaging servermay also include or may have access to means for performing a step inwhich transcoding is performed based on the inserted mediacharacteristics and the actual or assumed multimedia capabilities of thereceiving terminal, without performing an analysis of the message todetermine media characteristics of the message relevant to decidingwhether transcoding is needed.

Also in accord with the fourth aspect of the invention, the system mayalso include a transcoding server, and may be further characterized inthat the messaging server is configured to send the message to thetranscoding server if transcoding is needed, and the transcoding serveris configured to use the inserted media characteristics to itself decideif transcoding is needed.

Also in accord with the fourth aspect of the invention, the system mayalso include a transcoding server, and may be further characterized inthat the messaging server is configured to send the message to thetranscoding server if transcoding is needed, and the transcoding serveris configured to use the inserted media characteristics to itself decidewhich parts of the message need transcoding.

Also in accord with the fourth aspect of the invention, the system mayalso include a transcoding server, and may be further characterized inthat the messaging server is configured to determine, from the insertedmedia characteristics, which parts of the message need transcoding andto send the message to the transcoding server if transcoding is neededfor any message part, and to include in the message an indication ofwhich parts of the message need transcoding.

Also in accord with the fourth aspect of the invention, the system mayalso include a transcoding server, and may be further characterized inthat the means for transcoding is performed based on the inserted mediacharacteristics and the actual or assumed multimedia capabilities of thereceiving terminal, without performing an analysis of the message todetermine whether transcoding is needed.

In a fifth aspect of the invention, a computer program product isprovided comprising: a computer readable storage structure embodyingcomputer program code thereon for execution by a computer processor in asending terminal, said computer program code characterized in that itincludes instructions for performing the steps of a method according tothe first aspect of the invention and indicated as being performed bythe sending terminal.

In a sixth aspect of the invention, a computer program product isprovided comprising: a computer readable storage structure embodyingcomputer program code thereon for execution by a computer processor in amessaging server, said computer program code characterized in that itincludes instructions for performing the steps of a method according tothe first aspect of the invention and indicated as being performed bythe messaging server.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the inventionwill become apparent from a consideration of the subsequent detaileddescription presented in connection with accompanying drawings, inwhich:

FIG. 1 is a block diagram/flow diagram of a messaging system includingan external transcoding/adaptation server, according to the prior art.

FIG. 2 is a block diagram/flow diagram of a messaging system includingan external transcoding/adaptation server, according to the invention.

FIG. 3 is a flowchart showing the operation of a messaging systemaccording to the invention.

DETAILED DESCRIPTION

According to the invention, in order to enable a messaging server (e.g.an MMS Proxy-Relay in MMS, or an SIP proxy in SIP IM) to determinewhether transcoding/media adaptation is needed for a message intendedfor a receiving terminal, a user agent of the sending terminal insertscharacteristics of each of the media components of the message into themessage in a standard way (e.g. in the message header or body, accordingto a standard format). The messaging server examines only the insertedmedia characteristics information (like format profile, resolution,image-size, frame rate, etc) in deciding whether transcoding isnecessary, and thus need not examine/open any other part of the message,making it possible to get by with less processing complexity for themessaging and transcoding servers, and yet providing a decision as towhether transcoding is necessary in less time (at least for more complexmessages) than would be possible if the message had to be opened andexamined in its entirety. Also, the messaging server can make a decisionon transcoding without having knowledge of any given media format codingscheme (although it needs to know what media type and sometimes alsowhat media format is being used for the message components); themessaging server operative according to the invention only needs tocompare media characteristics with terminal capabilities (or adaptationtargets).

The characteristics information inserted into the message by the useragent include, in addition to the usual MIME (Multipurpose Internet MailExtensions) type information, more detailed information about theindividual media components of the message such as: image resolution,and format profile (e.g. JPEG baseline or progressive, H.263 profile andlevel information). In e-mail, SIP messaging and MMS, because of MIMEmultipart usage, there are already some headers having associated fieldsin which corresponding information is conveyed. These headers providesome of the information of use in determining whether transcoding isnecessary, headers such as the content-type header, which provides theMIME type of the message component, and the content-length header, whichindicates the length of the message component. According to theinvention, however, detailed characteristics such as image resolution,media profiles (JPEG baseline versus progressive, video profile andlevel), frame rate (for video clips), media contained in aMP4/3GPP/3GPP2 file format, and other comparable information areinserted in MIME multipart headers, for example in the content-typefield of the content-type header. Such insertions enable a messagingserver to know the media properties of the incoming message. For exampleif a SIP IM is sent carrying a JPEG image of resolution 480×320, thecontent-type header field could be:

Content-type: image/JPEG; media-pix-x=480; media-pix-  y=320;profile=baseline;Content-ID=<img.jpg>.Such a content-type header lets the messaging server know that theincoming message is of type JPEG baseline with a resolution of 480×320.If the terminal to which the message is intended is able to display only160×120, then the messaging server knows from the insertedcharacteristics and without otherwise opening the message (i.e. withoutexamining each media component of the message) that the message must betranscoded. It also knows which parts of the message needs to betranscoded and could decide to send only those parts for adaptation(either internally or to an external transcoding server).

Thus, and now referring to FIG. 1, according to the prior art a sendingterminal 11 sends a message to a receiving terminal 15, and in thecourse of delivery the message is processed by a messaging server 12.The messaging server—and more specifically a message router 12 aincluded as part of (i.e. in that it is hosted by) the messaging server12—sends a request for adaptation to a transcoding server 14 includingthe message, along with the capabilities of the receiving terminal orits identity (e.g. User-Agent header). Given the identity of thereceiving terminal, the transcoding server 14 is usually able to look upthe capabilities of the receiving terminal using an internal terminalcapability database. The transcoding server 14—and more specifically amessage analyzer and transcoding engine 14 a included as part of (hostedby) the transcoding server—then opens the message, examines each mediacomponent, and determines what if any transcoding is necessary, which itthen performs. It then sends the possibly transcoded message back to themessaging server, which forwards it to the receiving terminal. (In someprior art, instead of the message router 12 a, there is a messageadapter that opens the message and analyzes each part to determine whichparts need adaptation and then sends only those to the transcodingserver 14.)

In contrast, and now referring to FIG. 2, according to the invention, asending terminal 21 includes a user agent 21 a that inserts into amessage to be sent to the receiving terminal 25, according to one oranother convention such as described below, media characteristicsinformation about the message sufficient in detail to determine whethertranscoding is needed, given the capabilities of the receiving terminal.The sending terminal then sends the message to the receiving terminal25, and in the course of delivery the message is processed by amessaging server 22 according to the invention, i.e. a messaging server22 including a characteristics reader and analyzer module 22 a fordetermining whether there is a need for adaptation/transcoding of amessage based on the media characteristics inserted into the message.Prior to message delivery to the receiving terminal 25, the messagingserver 22 looks up the capabilities of the receiving terminal or somehowotherwise receives a specification of the capabilities, or else assumes(typically) some minimum set of multimedia capabilities, and then, usingthe characteristics reader and analyzer module 22 a, the messagingserver examines the inserted characteristics, and if transcoding isneeded based on the actual or assumed multimedia capabilities of thereceiving terminal 25, sends the message to a transcoding server 24 alsoaccording to the invention, i.e. the transcoding server would take intoaccount the media characteristics information of the message whendetermining which components of the message require adaptation. Thetranscoding server can itself look up the capabilities of the receivingterminal or the messaging server can include the capabilities with therequest for transcoding. In some embodiments, the transcoding server 24will itself re-examine the message and determine the transcoding needed,just as in the prior art, so that what the invention accomplishes insuch embodiments is to avoid having the messaging server 22 send amessage to an external transcoding server 24 when transcoding is notneeded. Also, since in such embodiments it is advantageous to have themessaging server be overly cautious in determining whether transcodingis needed, it is possible for the transcoding server 24 to decide thatno transcoding is needed even though the messaging server has decidedotherwise (such as e.g. if the message contains media components of atype the messaging server is not programmed to handle and thereforecannot determine whether the components are suitable for the receivingterminal without transcoding).

After transcoding (if needed), the transcoding server 24 sends the(possibly) transcoded message back to the messaging server, whichforwards it to the receiving terminal.

Note that the transcoding engine 24 a may actually be hosted by themessaging server 22, and not by a separate transcoding server, althoughfor purposes of a more plain comparison with the prior art asillustrated in FIG. 1, the transcoding engine 24 a is shown hosted by aseparate transcoding server 24, operated typically by a party differentthan the party operating the messaging server 22.

In the course of transcoding, the transcoding server 24 advantageouslymodifies the message's media characteristics information, which itincludes in the transcoded message it sends back to the messaging server22. In some embodiments, however, the media characteristics informationis stripped from the message. In such embodiments, if the receivingterminal 25 later forwards the message to another terminal, a user agentincluded in that terminal, acting now as a sender, inserts again messagemedia characteristics information in the message determined by analysingthe media components of the message. The messaging server 22 can alsotherefore behave differently in different embodiments: in some, itstrips the inserted media characteristics information and in others itretains the inserted media characteristics information.

The media characteristics can advantageously all be inserted into one ormore of the headers in the message header, rather than in the messagebody, in order to reduce the parsing time to extract such information.

Media Content Characteristics

In some embodiments, the media characteristics to be inserted into amessage by a user agent can include the media content characteristicsindicated in Table 1.

TABLE 1 Media content characteristics. Media content Media types tocharacteristic Description which it applies Charset Character set used(RFC 2045). Values include US-ASCII and Text content ISO-8859-1. ProfileProfile of the media component (e.g. Profile 0 for H.263 or All baselinefor JPEG). Level Level of the media component (e.g. Level 10 for H.263).All Media-pix-x Horizontal resolution of the visual media component(e.g. 640 Visual content pixels). Note that for video, conformance canoften be deduced from profile/level information but this mediacharacteristic provides more detailed information and could be useful.Media-pix-y Vertical resolution of the visual media component (e.g. 480Visual content pixels). Note that for video, conformance can often bededuced from profile/level information but this media characteristicprovides more detailed information and could be useful. Nb-frames Numberof frames in the visual media component (e.g. animated Visual contentGIF or video clip). This is optional. Frame-rate Frame-rate of thevisual media component (e.g. 15 frames per Visual content (e.g.seconds). video) Sampling-rate Sampling rate of the audio mediacomponent (e.g. 8000 Hz). Audio content This can also be implicit to theContent type. For instance audio/amr is 8000 Hz. Nb-channels Number ofchannels of the media component (e.g. stereo is 2, All mono is 1). Thiscan also be implicit to the Content type. For instance audio/amr ismono. Duration Approximate duration of the media component inmilliseconds All (e.g. duration of an audio or video clip, or apresentation). This is typically optional. Media-size The size of themedia component itself i.e. without MIME All headers and encoding fortransport (e.g. 5200 Bytes). Content-subtype The Content type of a mediacomponent embedded in a file File formats format (e.g. video/H263-2000and audio/AMR subtypes can be embedded in a video/3gpp file format).Subsequent content characteristics apply to this content-subtype (untila new content- subtype is listed). Content characteristics listed priorto all content-subtype apply to the Content-type itself as a whole.Content-ID Content-ID as described in RFC-2392 providing a reference toAll the body part to which the content description applies.X-MMS-Content- The list of Content Classes to which the MultimediaMessage Multimedia Message Class belongs to: TX = Text, IB = Imagebasic, IR = Image rich, VB = Video basic, VR = Video rich. E.g. TX, IB,IR. This media content characteristic is typically located in themessage header. X-Content- Description of the content present in themessage body. The Multimedia Message Description syntax is the same asfor the Content-type characteristic. In the case of multipart content, aline containing the Content- Description for each part shall beprovided. This media content characteristic is located in the messageheader.

Table 2 describes content-specific values of profiles and level mediacontent characteristics. For instance, Baseline and Progressive areacceptable JPEG profiles.

TABLE 2 Content-specific values of profile and level media contentcharacteristics. Content-type Profile Description Level DescriptionImage/jpeg Baseline Baseline DCT, non-differential, N/A Huffman coding,as defined in table B.1, symbol ‘SOF0’ in “MMS v1.2 ClientTransactions,” by the Open Mobile Alliance (OMA). ProgressiveProgressive DCT, non-differential, N/A Huffman coding, as defined intable B.1, symbol ‘SOF2’ in “MMS v1.2 Client Transactions,” by the OpenMobile Alliance (OMA). Video/h263-2000 0 through 10 (0 H.263 profilenumber specifying the 0-100 Level of bitstream and 3 are supported H.263annexes/subparts. operation specifying typically used H.263 Baselinecorresponds to Profile the level of for MMS) 0, Level 10. (See RFC3555.) computational complexity of the decoding process. H.263 Baselinecorresponds to Profile 0, Level 10. (See RFC 3555.) Application/smilSMIL-CONF-1-2 Indicates one or more base sets of SMIL modules that theclient supports. “SMIL-CONF-1-2” identifies the SMIL base set andassociated limitations defined in “MMS Conformance Document,” version2.0.0, February 2002, by CMG, Ericsson, Nokia, Sony- Ericsson, Comverse,Logica, Siemens, and Motorola; and in “MMS Conformance Document,”version 1.1, August 2001, by Ericsson, Nokia. SMIL-3GPP-R4 Predefinedvalues for base sets defined in “Transparent end-to-end Packet-switchedStreaming Service (PSS); Protocols and codecs,” Third GenerationPartnership Project, 3GPP TS 26.234, Rel 4. SMIL-3GPP-R5 Predefinedvalues for base sets defined in “Transparent end-to-end Packet-switchedStreaming Service (PSS); Protocols and codecs,” Third GenerationPartnership Project, 3GPP TS 26.234, Rel 5.

Table 3 describes MPEG-4 Visual Profile values of profile-level-id mediacontent characteristic.

TABLE 3 Content-specific values of profile and level media contentcharacteristics. Content-type Profile-level-id Description Video/mp4v-esDecimal value (a value A decimal representation of MPEG-4 Visual Profileand Level of 1 specifies MPEG-4 indication value(profile_and_level_indication) defined in Table G-1 Visual SimpleProfile of ISO/IEC 14496-2. See ISO/IEC 14496-2: 1999, “InformationLevel 1) technology - Coding of audio-visual objects - Part2: Visual,”and see ISO/IEC 14496-2: 1999/Amd.1:2000, “Information technology -Coding of audio-visual objects - Part 2: Visual, Amendment 1: Visualextensions.”

Representation of Media Content Characteristics in Messages

The invention encompasses many ways to represent media contentcharacteristics in a message. Among the many encompassed is anembodiment in which the media content characteristics are represented asparameters of the MIME content-type header field for the body partsspecified in RFC 2045. In such an embodiment, the media contentcharacteristics are part of the message body, and indicated inbold-italic in Message Fragment 1, which is an example of an MMS messageafter alteration (insertion of media characteristics) by the user agentin the sending terminal 21.

Message Fragment 1: Media Content Characteristics Represented asParameters of the MIME Content-Type Header Field, i.e. as Content-TypeParameters in Message Body.

MMS Headers X-Mms-Message-Type: M-send.req X-Mms-Transaction-ID:0123456789 X-Mms-MMS-Version: 1.3 From: +123/TYPE=PLMN To:+456/TYPE=PLMN Subject: My holidays Content-type:application/vnd.wap.multipart.related; type = “application/smil”; start= “<0000>”; Message Body Content-type: application/smil;profile=SMIL-3GPP-R5 Content-ID: <0000> .... SMIL presentation file....Content-type: image/jpeg; media-pix-x=640; media-pix-y=480;profile=Baseline; media-size=32768 Content-ID: <image1.jpg> ....jpeg-image.... Content-type: text/plain; charset=“us-ascii”;media-size=512 Content-ID: <text.txt> .... text.... Content-type:audio/amr; media-size=5000 Content-ID: <audio.amr> .... audio....Content-type: video/3gpp; media-size=44000;Duration=10000;Content-subtype-video/h263-2000; Profile=0; level=10; Frame-rate=15;media-size=40000; Content-subtype=audio/amr; media-size=4000 Content-ID:<video.3gp> .... audio+video clip....

In another embodiment, the media content characteristics are representedas parameters of the proposed content-description header field part ofthe message header. This is illustrated in Message Fragment 2. Thismethod has the advantage that one doesn't need to parse through themessage body to extract the relevant information which can improveperformance furthermore. Also, it allows global description of thecontent. For instance, it allows indicating to which MMS message classesthe message belongs to (see X-Mms-Content-Class descriptor). The contentdescription is linked to the body part through the use of the Content-IDparameter.

Message Fragment 2: Example of MMS Message with Media Characteristics asContent-Description Parameters in Message Header.

MMS Headers X-Mms-Message-Type: M-send.req X-Mms-Transaction-ID:0123456789 X-Mms-MMS-Version: 1.3 From: +123/TYPE=PLMN To:+456/TYPE=PLMN Subject: My holidays Content-type:application/vnd.wap.multipart.related; type = “application/smil”; start= “<0000>”; X-Mms-Content-Class = ”VB,VR” X-Content-Description:application/smil; profile=SMIL-3GPP-R5; Content-ID=<0000>X-Content-Description: image/jpeg; media-pix-x=640; media-pix-y=480;profile=Baseline; media-size=32768; Content-ID=<image1.jpg>X-Content-Description: text/plain; charset=“us-ascii”;media-size=512;Content-ID=<text.txt> X-Content-Description: audio/amr;media-size=5000;Content- ID=<audio.amr> X-Content-Description:video/3gpp; media-size=44000; Duration=10000;Content-subtype-video/h263-2000; Profile=0; level=10; Frame-rate=15;media-size=40000; Content-subtype=audio/amr; media-size=4000;Content-ID=<video.3gp> Message Body Content-type: application/smilContent-ID: <0000> .... SMIL presentation file.... Content-type:image/jpeg Content-ID: <image1.jpg> .... jpeg-image.... Content-type:text/plain; charset=“us-ascii” Content-ID: <text.txt> .... text....Content-type: audio/amr Content-ID: <audio.amr> .... audio....Content-type: video/3gpp Content-ID: <video.3gp> .... audio+videoclip....

An essentially equivalent embodiment is one in which the mediacharacteristics are represented using the content-features header in amanner described in RFC 2912, as shown in Message Fragment 3.

Message Fragment 3: Example of MMS Message with Media Characteristics asContent-Feature Parameters in Message Body.

MMS Headers X-Mms-Message-Type: M-send.req X-Mms-Transaction-ID:0123456789 X-Mms-MMS-Version: 1.3 From: +123/TYPE=PLMN To:+456/TYPE=PLMN Subject: My holidays Content-type:application/vnd.wap.multipart.related; type = “application/smil”; start= “<0000>”; Message Body Content-type: application/smilContent-features: (profile=SMIL-3GPP-R5) Content-ID: <0000> .... SMILpresentation file.... Content-type: image/jpeg Content-features: ( &(media-pix-x=640)(media-pix- y=480)(profile=Baseline)(media-size=32768))Content-ID: <image1.jpg> .... jpeg-image.... Content-type: text/plain;charset=“us-ascii” Content-features: (media-size=512) Content-ID:<text.txt> .... text.... Content-type: audio/amr Content-features:(media-size=5000) Content-ID: <audio.amr> .... audio.... Content-type:video/3gpp Content-features: (& (media-size=44000)(Duration=10000)   (&(Content-subtype=video/h263-2000)(Profile=0)(level=10)(Frame-rate=15)(media-size=40000))   (&(Content-subtype=audio/amr)(media-size=4000))) Content-ID: <video.3gp>.... audio+video clip.... .... audio.... Content-type: video/3gppContent-features: (& (media-size=44000)(Duration=10000)   (&(Content-subtype=video/h263-2000)(Profile=0)(level=10)(Frame-rate=15)(media-size=40000))   (&(Content-subtype=audio/amr)(media-size=4000))) Content-ID: <video.3gp>.... audio+video clip....

Another possibility is to have the media content characteristics berepresented as parameters of the content-features as part of the messageheader, as indicated in Message Fragment 4. This method has theadvantage that one doesn't need to parse through the message body toextract the relevant information which can improve performancefurthermore. Also, it allows global description of the content. Forinstance, it allows indicating to which MMS message classes the messagebelongs to (see X-Mms-Content-Class descriptor).

Message Fragment 4: Example of MMS Message with Media Characteristics asContent-Feature Parameters in Message Header.

MMS Headers X-Mms-Message-Type: M-send.req X-Mms-Transaction-ID:0123456789 X-Mms-MMS-Version: 1.3 From: +123/TYPE=PLMN To:+456/TYPE=PLMN Subject: My holidays Content-type:application/vnd.wap.multipart.related; type = “application/smil”; start= “<0000>”; Content-features: ( & (Type=“application/smil”)(profile=SMIL- 3GPP-R5) (Content-ID=<0000>))Content-features: ( & (Type= “image/jpeg”)(media-pix-x=640)(media-pix-y=480) (profile=Baseline)(media-size=32768)(Content-ID=<image1.jpg>)) Content-features: ( & (Type=“text/plain”)(charset=us-ascii) (media-size=512)( Content-ID=<text.txt>)) Content-features: ( & (Type= “audio/amr”)(media-size=5000)(Content-ID=<audio.amr>)) Content-features: ( & (Type=“video/3gpp”)(Duration=10000)(media-size=44000)(Content-subtype=video/h263-2000)(Profile=0)(level=10)(Frame-rate=15)(media-size=40000)(Content-subtype=audio/amr)(media-size=4000) (Content-ID=<video.3gpp>))Message Body Content-type: application/smil Content-ID: <0000> .... SMILpresentation file.... Content-type: image/jpeg Content-ID: <image1.jpg>.... jpeg-image.... Content-type: text/plain; charset=“us-ascii”Content-ID: <text.txt> .... text.... Content-type: audio/amr Content-ID:<audio.amr> .... audio.... Content-type: video/3gpp Content-ID:<video.3gp> .... audio+video clip....

Note that it is possible to provide media characteristics in both themessage header and body. If there is a contradiction between them, whichshould not normally happen, a method to resolve the conflict can bedefined. For instance the message header information can be defined tohave precedence since it is likely to be the only information used by amessaging server if present and complete (i.e., if complete, it wouldnot even parse the message body).

Comparison of media characteristics with terminal capabilities

In order to determine if the message content is suitable for a giventerminal, the media characteristics present in the message header orbody must be compared with the terminal capabilities. The terminalcapabilities can be obtained through a wide variety of methods includingterminal capability databases, user terminal profile databases, andUAProf. For instance, Table 4 summarizes the attributes defined withinthe MMS Characteristics component in MMS v1.2. (See: “MMS v1.2 ClientTransactions,” Open Mobile Alliance (OMA), athttp://www.openmobilealliance.org).

TABLE 4 Terminal attributes defined within the MMS Characteristicscomponent in MMS v1.2. Resolution Sample Attribute Description Rule TypeValues Component: MMS Characteristics MmsMaxMessageSize The maximum sizeof a Locked Number 20480 multimedia message in bytes.MmsMaxImageResolution The maximum size of an image Locked Literal “80 ×60” in units of pixels (horizontal × vertical). MmsCcppAccept List ofsupported content types Locked Literal “image/jpeg”, conveyed as MIMEtypes. bag “audio/wav”, “video/mpeg-4” MmsCcppAcceptCharSet List ofcharacter sets that the Locked Literal “US-ASCII”, MMS Client supports.Each bag “ISO-8859-1” item in the list is a character set nameregistered with IANA. MmsCcppAcceptLanguage List of preferred languages.Locked Literal “en”, The first item in the list should bag “fr” beconsidered the user's first choice. Property value is a list of naturallanguages, where each item in the list is the name of a language asdefined by RFC 1766. MmsCcppAcceptEncoding List of transfer encodingsthat Locked Literal “base64”, the MMS Client supports. bag “quoted-Property value is a list of printable” transfer encodings, where eachitem in the list is a transfer encoding name as specified by RFC 2045and registered with IANA. MmsVersion The MMS versions supported LockedLiteral “2.0”, “1.3” by the MMS Client conveyed bag asmajorVersionNumber.minorVersion Number. MmsCcppStreamingCapableIndicates whether the MMS Locked Boolean “Yes”, “No” Client is capableof invoking streaming. MmsSmilBaseSet Indicates one or more base LockedLiteral “SMIL-CONF- sets of SMIL modules that the bag 1-2” clientsupports. “SMIL-CONF- 1-2” identifies the SMIL base set and associatedlimitations defined in MMS CONF (MMS Conformance document of the OMA MMSspecifications v 1.2). Predefined values for base sets defined in3GPP TS26.234 may also be used (e.g. SMIL-3GPP-R and SMIL- 3GPP-R).X-Mms-Content-Class List of supported MM Content Locked Literal “TX”,“IB”, “IR”, Classes: bag “VB”, “VR” TX = Text; IB = Image basic; IR =Image rich; VB = Video basic; and VR = Video rich;MmsSuppressContentAdaptation Requests that MMS Proxy- Locked Boolean“Yes”, “No” Relay performs no content adaptation.

The messaging server determines the need for adaptation by comparing themedia characteristics with the terminal capabilities. Adaptation isrequired if: the media characteristics are not supported by the terminal(e.g. message size is too large or MMS message class not supported); orthe message contains any component for which the MIME content-type isnot supported by the terminal; or the message contains components forwhich the characteristics are not supported by the terminal and wouldcause interoperability problems (such as e.g. the resolution is toolarge for what the terminal can support, or the profile or level of thecomponent is not supported, and so on).

Table 5 indicates how media characteristics and terminal capabilitiescan be compared in the case of an MMS terminal in order to determinewhether transcoding (of an MMS message) is needed.

TABLE 5 Comparison of media characteristics and UAProf capabilities inMMS. Condition for the content to be suitable for the Mediacharacteristic UAProf descriptor terminal X-Mms-Content-ClassMmsContentClass The intersection between X-Mms-Content-Class mediacharacteristic and UAProf's MmsContentClass must be non-empty.Media-size at message level MmsMaxMessageSize Media-size at messagelevel must not exceed MmsMaxMessageSize. Content-type, Content-MmsCcppAccept All MIME type values of content-type, content- subtype,Content- subtype and content-description must be element of DescriptionMmsCcppAccept. Media-pix-x and Media-pix-y MmsMaxImageResolutionMedia-pix-x by Media-pix-y must not exceed MmsMaxImageResolution.Charset MmsCcppAcceptCharSet Charset value must be element ofMmsCcppAcceptCharSet. Content-Encoding MmsCcppAcceptEncodingContent-Encoding must be element of MmsCcppAcceptEncoding. Profile (forapplication/smil) MmsSmilBaseSet Profile of an application/smilcontent-type must be element of MmsSmilBaseSet.

Currently, MMS terminals do not report image and video relevantprofiles/capabilities (such as JPEG baseline or progressive, H.263profile and level). Thus, to implement the invention, MMS terminals needto be enhanced to provide such information so that the terminalcapabilities can be compared with media characteristics as describedabove. In case of an MMS terminal not so enhanced, and when terminalcapabilities are not otherwise available, the invention can be practisedassuming that the most basic profile and level is supported by theterminal (e.g. JPEG baseline or H.263 Profile 0 level 10 if video issupported). But when available, profile and level information (orprofile-level-id) should of course be compared.

Referring now to FIG. 3 (and still also to FIG. 2), the invention isshown as providing a method including a first step 31 in which thesending terminal's user agent 21 a inserts media characteristicsinformation in a message (after possibly analyzing each media component)intended for the receiving terminal 25. In a next step 32, the sendingterminal 21 sends the message to the receiving terminal 25, with theresult that the message arrives at the messaging server 22 en route tothe receiving terminal. In a next step 33, the messaging server readsthe inserted media characteristics, compares them with actual or assumedcapabilities of the receiving terminal (actual being obtained e.g. by alook up), and decides whether there is a need for any transcoding. Iftranscoding is not needed, then in an optional next step 37, themessaging server removes the inserted media characteristics (possiblybased on type of receiving terminal), and in a next step 38 themessaging server sends the message to the receiving terminal. If howevertranscoding is needed (according to the comparison made by the messagingserver), then in a next step 34 the messaging server sends the messageto the transcoding server 24 (assumed here to be external to themessaging server, but could also be hosted by messaging server) fortranscoding (along with an indication of the capabilities of thereceiving terminal or its identity information for use in possiblyobtaining the capabilities of the receiving terminal, as explainedabove). In a next step 35 the transcoding engine hosted by thetranscoding server transcodes the message based on the capabilities ofthe receiving terminal, possibly using the inserted mediacharacteristics as a guide to what needs to be transcoded in order tosave analysis. Then in a next step 36, the transcoding engine returnsthe message to messaging server. Then, optionally, the messaging serveroptionally performs step 37 in which it removes the inserted mediacharacteristics. Finally, the messaging server sends the message to thereceiving terminal (in step 38).

As explained above, the invention provides both a method andcorresponding equipment consisting of various modules providing thefunctionality for performing the steps of the method. The modules may beimplemented as hardware, or may be implemented as software or firmwarefor execution by a processor. In particular, in the case of firmware orsoftware, the invention can be provided as a computer program productincluding a computer readable storage structure embodying computerprogram code—i.e. the software or firmware—thereon for execution by acomputer processor.

It is to be understood that the above-described arrangements are onlyillustrative of the application of the principles of the presentinvention. In particular, the invention applies to a wide range ofmessaging servers including MMS Proxy-relays, SIP messaging servers,e-mail servers, and others. Also, the terminals are not limited towireless terminals but include PCs, PDAs, and other kinds of terminalsable to be used for communication. Further, the term multimedia messageshould be understood to include any message containing multimediacontent, including e.g. one or more of the following kinds of content:images, video, audio, and text, as well as other kinds of content. Inaddition, the transcoding server can be an external server to themessaging server or be an internal process handling transcoding withinthe messaging server. Finally, numerous modifications and alternativearrangements may be devised by those skilled in the art withoutdeparting from the scope of the present invention, and the appendedclaims are intended to cover such modifications and arrangements.

What is claimed is:
 1. A messaging server, comprising a processorconfigured to: obtain media characteristics of a multimedia messageinserted into the multimedia message intended for a receiving terminal;decide whether the multimedia message should be transcoded based only oncomparing the media characteristics of the multimedia message with atleast one of actual multimedia capabilities and assumed multimediacapabilities of the receiving terminal; and remove the mediacharacteristics that are inserted into the multimedia message beforesending the multimedia message to the receiving terminal whentranscoding is not needed.
 2. The messaging server of claim 1, whereinthe multimedia message includes a header portion and a body portion. 3.The messaging server of claim 2, wherein the media characteristics ofthe multimedia message are inserted into a field in the header portionof the multimedia message.
 4. The messaging server of claim 1, theprocessor further configured to: determine, from the inserted mediacharacteristics, which at least one part of the message needstranscoding; and send only the at least one part requiring transcodingto a transcoding server.
 5. The messaging server of claim 1, theprocessor further configured to send the multimedia message to atranscoding server.
 6. The messaging server of claim 5, wherein thetranscoding server is configured to use the inserted mediacharacteristics to decide if transcoding is needed.
 7. The messagingserver of claim 1, the processor further configured to: determine, basedon the inserted media characteristics, parts of the multimedia messageneeding transcoding; send the multimedia message to a transcoding serverif transcoding is needed for any message part; and include, in themultimedia message, an indication of which parts of the multimediamessage need transcoding.
 8. A method comprising: obtaining mediacharacteristics of a multimedia message inserted into the multimediamessage intended for a receiving terminal; deciding whether themultimedia message should be transcoded based only on comparing themedia characteristics of the multimedia message with actual or assumedmultimedia capabilities of the receiving terminal; and removing themedia characteristics that are inserted into the multimedia messagebefore sending the multimedia message to the receiving terminal whentranscoding is not needed.
 9. The method of claim 8, wherein themultimedia message includes a header portion and a body portion.
 10. Themethod of claim 9, wherein the media characteristics of the multimediamessage are inserted into a field in the header portion of themultimedia message.
 11. The method of claim 8 further comprising sendingthe multimedia message to a transcoding server.
 12. The method of claim8, further comprising: determining, from the inserted mediacharacteristics, which at least one part of the message needstranscoding; and sending only the at least one part requiringtranscoding to a transcoding server.
 13. The method of claim 8 furthercomprising transcoding the multimedia message based on the mediacharacteristics and multimedia capabilities of the receiving terminal.14. The method of claim 13, wherein the transcoding is done withoutperforming an analysis of the multimedia message to determine mediacharacteristics of the multimedia message relevant to deciding whethertranscoding is needed.
 15. The method of claim 13 further comprisingremoving the media characteristics from the multimedia media messageafter transcoding when transcoding is needed and before sending themultimedia message to the receiving terminal.
 16. The method of claim 8,wherein the media characteristics of the multimedia message are insertedinto a header portion of the multimedia message.
 17. A computer programproduct comprising a non-transitory computer-readable storage mediumhaving computer-readable program code stored therein, which in responseto execution by at least one processor, causes an apparatus to: obtainmedia characteristics of a multimedia message inserted into themultimedia message intended for a receiving terminal; decide whether themultimedia message should be transcoded based only on comparing themedia characteristics of the multimedia message with at least one ofactual multimedia capabilities and assumed multimedia capabilities ofthe receiving terminal; and remove the media characteristics that areinserted into the multimedia message before sending the multimediamessage to the receiving terminal when transcoding is not needed. 18.The computer program product of claim 17, wherein execution by theprocessor causes the apparatus to send the multimedia message to thetranscoding server.
 19. The computer program product of claim 18,wherein the transcoding is done without performing an analysis of themultimedia message to determine media characteristics of the multimediamessage relevant to deciding whether transcoding is needed.
 20. Thecomputer program product of claim 17, wherein execution by the processorcauses the system to: determine, from the inserted mediacharacteristics, which at least one part of the message needstranscoding; and send only the at least one part requiring transcodingto a transcoding server.
 21. The computer program product of claim 17,wherein the apparatus includes a user agent and execution by theprocessor causes, at the user agent, inserting the media characteristicsinto the multimedia message to enable determining whether the multimediamessage should be transcoded to accommodate multimedia capabilities ofthe receiving terminal.